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In an attempt to replicate an earlier study by H. G, 
Schmidt, H. P. A. Boshuizen, and P. P. M. Hobus (1988), 96 students 
and physicians at the University of Limburg (Netherlands) studied 4 
clinical cases, produced a diagnosis, and recalled each case. 
Subjects were 24 second-year, 24 fourth-year, and 24 sixth-year 
medical students and 24 internists with at least 4 years of 
experience in internal medicine. In addition, processing time was 
manipulated. Diagnostic accuracy increased with level of expertise. 
Unexpectedly, the study failed to disclose an intermediate effect in 
the recall data. A positive, linear relationship between expertise 
level and case recall was found. Constraining processing time did not 
affect diagnostic accuracy, but equally affected the recall 
performance of the various expertise levels. The results are 
interpreted in terms of the inability of even expert physicians to 
process laboratory data in an automatic fashion. Two figures 
illustrate the report. (Contains 16 references.) (SLD) 
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Abstract 



The present study attempted to replicate an earlier study by Schmidt, Boshuizen, 
& Hobus (1988). Subjects studied four clinical cases, produced a diagnosis and 
recalled each case. In addition, processing time was manipulated. Diagnostic 
accuracy increased with level of expertise. Unexpectedly, the study failed to 
disclose an intermediate effect in the recall data: a positive, linear relation 
between expertise level and case recall was found. Constraining processing time 
did not affect diagnostic accuracy, but equally affected the recall performance of 
the various expertise levels. The results are interpreted in terms of the inability 
of even expert physicians to process laboratory data in an automatic fashion. 



ERLC 



3 



Theories of text-processing generally assume that prior knowledge facilitates new 
information encoding and retrieval (e.g., Bransford & Johnson, 1972). New 
information from text and prior knowledge interact in the resulting mental 
representation. Studies using free recall of meaningful texts assume that the 
recall reflects content and structure of the mental representation of the text. 
Since experts have access to more elaborate and more coherent knowledge 
structures they are conjectured to recall more information from the text than 
novices. Research on expertise in domains such as chess (De Groot, 1965), 
baseball (Spilich, Vesonder, Chiesi, & Voss, 1979), bridge (Chamess, 1979), and 
computer-programming (McKeithen, Reitman, Rueter, & Hirtle, 1981) support 
this hypothesis. 

Studies on expertise in the domain of medicine, however, do not produce 
such a clear-cut linear relationship. Only few recall studies have demonstrated a 
positive relationship with level of expertise (Hobus, Schmidt, Boshuizen, & 
Patel, 1987; Norman, Brooks, & Allen, 1989). Quite a number of researchers have 
failed to find significant effects of expertise level on the amount of information 
recalled, although effects were found on related measures such as recall per imit 
study time or recall time (e.g., Claessen & Boshuizen, 1985; Norman, Jacoby, 
Feightner, & Campbell, 1979). Finally, a nvimber of other studies demonstrated 
an inverted U-shaped relation between level of expertise and amount of clinical 
case recall, the so-called "intermediate effect" (e.g., Gilhooly, McGeorge, Hunter, 
Rawles, Kirby, Green, Wymi, & Simpson, 1992; Patel & Groen, 1991; Patel & 
Medley-Mark, 1986; Schmidt & Boshuizen, in press; Schmidt, Boshuizen, & 
Hobus, 1988). A meta-analysis of outcomes of 12 recall studies conducted 
between 1979 and 1989 suggests that the inverted U-shaped relation better 
describes the combined results than a monotonically increasing function 
(Boshuizen, 1989). Schmidt and Boshuizen (in press) hypothesized that this 
decline in recall performance beyond the intermediate level might result from 
different ways of case processing due to knowledge restructuring. In diagnosing a 
case students have to reason through their rich, elaborated causal networks 
explaining the causes and consequences of disease in terms of general underlying 
biological or pathophysiological processes. Through extensive and repeated 
application of acquired knowledge, and particularly through exposure to patient 
problems, clusters of lower order concepts in these causal networks become 
encapsulated into some higher level concepts or diagnostic labels. Schmidt and 
Boshuizen found empirical evidence for this assumption by experimentally 
manipulating case processing in medical students and experts. 
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The 1988 study by Schmidt and colleagues plays a crucial role in the theory 
by Schmidt, Norman and Boshuizen (1991) about the development of medical 
expertise. In this study it was found that tnedical experts remembered less from a 
case than advanced students when given ample time to process a case; yet their 
diagnostic performance and their recall were not affected by a sizable reduction in 
processing time, whereas the students' performance in fact was. Because of this 
central role of this phenomenon in their theory, we decided to replicate this 
study. However, instead of one, four cases were presented, taken from four 
different specialty areas in internal medicine. 



Method 

Subjects. Subjects were 96 students and physicians of the University of 
Limburg: 24 second-year, 24 fourth-year and 24 sixth-year medical students and 24 
internists with at least 4 years of experience in internal medicine. Each group of 
24 was randomly subdivided Into three groups of eight who were assigned to 
three time constraint conditions. Subjects received a small compensation for 
their participation. 

Material The materials consisted of four booklets, each containing a 
description of a clinical case and two blank response sheets. Each clinical case 
description reported some contextual information and complaint, the case 
history, the physical examination, the relevant laboratory data and some 
additional findings. The case descriptions were about half a page in length and 
consisted of 33, 42, 43, and 35 propositions respectively. The four clinical cases, 
pheohromocytoma, stomach carcinoma, heart failure and liver cirrhosis, were 
based on actual patients and were presented as a narrative. 

Procedure. Subjects were told that four cases would be presented and that it 
would be their task to produce a diagnosis and to write down what they 
remembered from the case. Then an example case was presented in order to 
make them familiar with the case format in this experunent and to experience 
the reading time allowed. Depending on the experimental condition subjects 
were given the opportunity to read each case for 3 minutes (3'00"), 1 minute and 
15 seconds (115")/ or 30 seconds (30"). Subjects were free to use as much time as 
they needed for the assignments. The order of case presentation was balanced. 

Analysis. Diagnoses were scored on a scale ranging from 0 (incorrect 
diagnosis) to 6 (completely correct diagnosis) for each case. Recall protocols were 
segmented into small meaningful information units referred to as propositions. 
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The number of correct propositions was tallied. Reliabilities of these procedures 
exceeded .98. Manova with repeated measures was used to analyze the data. 



Results and discussion 



Figure 1 shows diagnostic accuracy as a function of level of expertise and 
processing time. A significant main effect of level of expertise on diagnostic 
accuracy was found, F(3,84) = 68,68, p = .0001. An increase in diagnostic accuracy 
is considered one of the most stable effects of medical expertise. Therefore, we 
can conclude that the subjects' tasks in this experimental setting were 
ecologically valid. The effect of processing time was not significant F(3,84) = .29, p 
= .75. The same effect was found by Schmidt et al. (1988). This suggests that 
processing a clinical case in order to formulate a diagnosis directs the attention of 
the subjects in such a way that they are equally capable or incapable of fulfilling 
this task in 3'.00", 1'.15" or 30". 
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Figure 1. Average accuracy of diagnoses as a fimction of expertise level and 
processing time. 
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The graph depicting the relation between average number of propositions 
recalled, level of expertise and processing time (see Figure 2) shows a positive 
linear recall function with expertise, F(3,84) = 10.27, p = .0001. In addition the 
main effect of processing time is significant as well, F(3,84) = 59.89, p = .0001, but 
no interaction effect of processing time and level of expertise occurred. 
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Figure 2. Average number of propositions recalled as a function of expertise 
level and processing time. 



These results clearly contradict the results of the Schmidt et al. (1988) shidy. 
There it was found that medical experts remembered less from a case than 
medical students in the long processing time condition. The experts' recall, 
however, was not affected by constraining processing time, whereas the students- 
recall was. The question, then, is how the discrepancies between the present 
study and the Schmidt et al. study can be explained. 

First, the present experiment used four cases instead of one. There is some 
evidence that experts consider the clinical case representation task initially as a 
problem-solving task (Schmidt & Boshuizen, in press), making recall 
performance incidental to that task. However, presentation of four cases may 
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lead them to adapt to the requirements of the experiment and improve their 
performance on later cases. Analysis of the effect of presentation order on the 
number of case propositions recalled accurately showed indeed a significant 
effect, F(3, 357) = 2.998, p = .03. This position effect, however, can not completely 
account for the failure to replicate the intermediate effect, since recall 
performance on the first case also showed a significant, positive, linear 
relationship with level of expertise. 

Further procedures and conditions of both experiments were the same 
except that different cases were used. Therefore a second explanation for the 
discrepancy must be sought in the main difference between the cases, which is 
the presence of numerical lab data in the cases of the present study. The bacterial 
endocarditis case used by Schmidt et al. contained virtually no lab data, whereas 
all four cases in the present study contained approximately 25% lab data in a 
numerical form. Studies of numerical lab data recall by Norman et al (1989) 
indicate that, in those cases, even expert physicians cannot built up a mental 
representation of the clinical problem at hand automatically and immediately, 
but have to process the lab data in an analytical fashion, combining data from 
various tests. This analytical approach to the data may lead to better recall (which 
was what Norman and his collaborators found). If the processing of lab data 
proceeds in a different mode than the processing of other signs and symptoms 
presented in a case, the failure to find an intermediate effect in the present study 
may be explained. This possibility is presently under study in our lab. 
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